System and method for controlling vehicle

ABSTRACT

Provided is a system and method for controlling a vehicle. The vehicle control system includes an input unit configured to collect driving situation data and driver&#39;s state data, a memory configured to store a program for determining a driving pattern using the driving situation data and the driver&#39;s state data in the case of an autonomous driving mode, and a processor configured to execute the program. The processor learns the driving situation data and the driver&#39;s state data to determine a driver&#39;s preferred driving pattern and transmit an autonomous driving control command according to the driving pattern.

CROSS-REFERENCE TO RELATED APPLICATION

This application claims priority to and the benefit of Korean PatentApplication No. 10-2020-0169822, filed on Dec. 7, 2020, Korean PatentApplication No. 10-2020-0169823, filed on Dec. 7, 2020, Korean PatentApplication No. 10-2020-0169824, filed on Dec. 7, 2020, the disclosureof which are incorporated herein by reference in their entirety.

BACKGROUND 1. Field

The present disclosure relates to a system and method for controlling avehicle.

2. Discussion of Related Art

Vehicle driving modes may be divided into an autonomous driving mode inwhich a vehicle has a driving control right and a driver driving mode inwhich a driver has a driving control right.

In the case of an autonomous driving mode, according to the related art,a control for autonomous driving is performed in consideration of aroute to a destination and surrounding conditions (obstacle information,signal information, and the like), but there is a limitation in notconsidering the preferences and characteristics of individual drivers indetermining the autonomous driving pattern.

In the case of a driver driving mode, a head-up display (HUD) devicethat graphically displays a vehicle speed, a remaining fuel amount, roadguidance, advanced driver-assistance system (ADAS) information, etc. ona windshield in front of a driver is applied. The HUD according to therelated art does not consider a driver's gaze position (gaze point), andtherefore, there is a problem in that a driver cannot recognizeinformation displayed on the HUD depending on the situation, and the HUDcannot be controlled in consideration of a driver's condition andillumination information.

SUMMARY

The present disclosure proposed to solve the above-mentioned problemsand an object of the present disclosure provide a system and method forcontrolling a vehicle capable of performing learning-type autonomousdriving control in consideration of a driver's state, a driving history,or a pattern in an autonomous driving mode, determining a driver's faceorientation and gaze to control head-up display (HUD) information to beprovided at a location where a gaze point is located, supportingintuitive recognition of the HUD information even when the driver's gazemoves, reducing a driver fatigue in consideration of the driver'scondition and illumination information, and controlling the HUD so thatvisibility of the HUD is improved.

According to an aspect of the present disclosure, there is provided avehicle control system including an input unit configured to collectdriving situation data, infrastructure data, forward situation data, anddriver's state data, a memory configured to store a program fordetermining a driving pattern using the driving situation data and thedriver's state data in a case of an autonomous driving mode, and aprocessor configured to execute the program, in which the processorlearns the driving situation data, the infrastructure data, the forwardsituation data, and the driver's state data to determine a driver'spreferred autonomous driving pattern, and transmit an autonomous drivingcontrol command according to the driving pattern.

The driving situation data may include speed, acceleration, yaw, pitch,and roll information and steering angle information of a vehicle.

The infrastructure data may include information on weather, a pavementcondition, and a road structure.

The forward situation data may include information on whether a frontvehicle is present, a color and a size of the front vehicle, whether agaze point overlaps a head-up display (HUD) position and a lightingdevice of the front vehicle, and the like.

The driver's state data may include gaze direction information, eyeblink information, pupil diameter information, face positioninformation, and expression information of a driver.

The processor may perform learning by matching the autonomous drivingdata, the driving situation data, and the driver's state data accordingto time sequence, and use the learned result to transmit an autonomousdriving control command according to the driver's preferred drivingpattern.

The processor may include at least one of a steering wheel controlcommand for a turning radius on a curved road, a vehicle speed controlcommand, an acceleration control command, and a command for maintaininga separation distance from the front vehicle.

In a case of a driver driving mode, the memory may further store asecond program for determining an HUD control command using the driver'sstate data including driver's gaze position information and the drivingsituation data including position and color information of a surroundingobject, and the processor may execute the second program and transmitthe HUD control command in consideration of the color information of thesurrounding object displayed in an HUD area in the driver's gaze.

The processor may transmit a control command for changing a color of ahead-up display user interface (HUD UI) to the HUD so that a color ofcontent displayed in the HUD area is contrasted/compared with the colorof the surrounding object to maximize day/night visibility/readability.

The processor may determine the gaze point using the driver's gazeposition information and transmit a control command to display HUDinformation at the gaze point.

In a case of a driver driving mode, the memory may further store a thirdprogram for determining an HUD control command by using at least one ofthe driver's state data related to a driver fatigue and the drivingsituation data related to illuminance, and the processor may execute thethird program and transmit the HUD control command for changing at leastone of color, brightness, and saturation of content displayed on theHUD.

According to another aspect of the present disclosure, there is provideda vehicle control method, including (a) collecting driver's state dataand driving state data, (b) in an autonomous driving mode, performinglearning using autonomous driving control data, the driver's state data,and the driving state data, and determining a driver's preferred drivingpattern, and (c) transmitting an autonomous driving control commandaccording to the driving pattern.

In operation (a), the driving situation data including vehicle speed,acceleration, yaw, pitch, and roll information and steering angleinformation, and the driver's state data including gaze directioninformation, eye blink information, pupil diameter information, faceposition information, and expression information of a driver may becollected.

In operation (b), the driving pattern including at least one of asteering wheel control command for a turning radius on a curved road, avehicle speed control command, an acceleration control command, and acommand for maintaining a separation distance from a front vehicle maybe determined.

The vehicle control method may further include, (d) when the autonomousdriving mode is switched to a driver driving mode, acquiring driver'sgaze position information and information on a position, a color, andluminance of a surrounding object corresponding to the position, anddetermining an HUD control command.

In operation (d), the color and luminance of the content displayed in anHUD area may be contrasted/compared with the color of the surroundingobject in consideration of the color and luminance information of thesurrounding object displayed in the HUD area in the driver's gaze andmay transmit the HUD control command to maximize day/nightvisibility/readability.

The vehicle control method may further include (e) when the autonomousdriving mode is switched to the driver driving mode, determining an HUDcontrol command by using at least one of the driver's state data relatedto a driver fatigue and the driving situation data related toilluminance.

In operation (e), the HUD control command for changing at least one ofcolor, brightness, and saturation of content displayed on the HUD may bedetermined.

The above-described configurations and operations of the presentdisclosure will become more apparent from embodiments described indetail below with reference to the drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

The above and other objects, features and advantages of the presentdisclosure will become more apparent to those of ordinary skill in theart by describing exemplary embodiments thereof in detail with referenceto the accompanying drawings, in which:

FIG. 1 illustrates a vehicle control system according to an embodimentof the present disclosure;

FIG. 2 illustrates a vehicle control apparatus according to anembodiment of the present disclosure;

FIG. 3 illustrates autonomous driving improvement in consideration of adriver's state according to an embodiment of the present disclosure;

FIG. 4 illustrates a vehicle control method according to an embodimentof the present disclosure;

FIG. 5A and FIG. 5B illustrate an example of a change of HUD informationdisplay area in consideration of driver's field of view information anda safety-related important information display position on an HUD screenlayout according to another embodiment of the present disclosure;

FIG. 6 illustrates a vehicle control method according to anotherembodiment of the present disclosure;

FIG. 7A, FIG. 7B, and FIG. 7C illustrate a concept of HUD control usinga driver's state, surrounding information, infrastructure information,and forward situation information according to still another embodimentof the present disclosure; and

FIG. 8 illustrates a vehicle control method according to yet anotherembodiment of the present disclosure.

DETAILED DESCRIPTION OF EXEMPLARY EMBODIMENTS

The above-described objects and other objects of the present disclosureand methods accomplishing them will become apparent from the followingdescription of embodiments with reference to the accompanying drawings.

However, the present disclosure is not limited to the embodimentsdisclosed below but may be implemented in various different forms. Onlythe following embodiments are provided to easily inform those ofordinary skill in the art to which the present disclosure pertains thatthe object, configuration and effect of the disclosure, and the scope ofthe present disclosure is defined by the description of the claim.

Meanwhile, terms used in the present specification are for describingthe embodiments rather than limiting the present disclosure. Unlessotherwise stated, a singular form includes a plural form in the presentspecification. Components, steps, operations, and/or elements mentionedby terms “comprise” and/or “comprising” used in the present disclosuredo not exclude the existence or addition of one or more othercomponents, steps, operations, and/or elements.

FIG. 1 illustrates a vehicle control system according to an embodimentof the present disclosure.

The vehicle control system according to the present disclosure includesan input unit 110 configured to collect driving situation data, driver'sstate data, infrastructure information, and forward situation data, amemory 120 configured to store a program for determining a drivingpattern using the driving situation data and the driver's state data inthe case of an autonomous driving mode, and a processor 130 configuredto execute the program, in which the processor 130 learns the drivingsituation data and the driver's state data to determine a driver'spreferred driving pattern and transmit an autonomous driving controlcommand according to the driving pattern.

The driving situation data includes speed, acceleration, yaw, pitch, androll information and steering angle information of a vehicle.

The driver's state data includes gaze direction information, eye blinkinformation, pupil diameter information, face position information, andexpression information of a driver.

The processor 130 performs learning by matching the autonomous drivingdata, the driving situation data, the driver's state data, theinfrastructure information, and the forward situation data according totime sequence, and uses the learned result to transmit an autonomousdriving control command according to the driver's preferred drivingpattern.

The processor 130 includes at least one of a steering wheel controlcommand for a turning radius on a curved road, a vehicle speed controlcommand, an acceleration control command, and a command for maintaininga separation distance from the front vehicle.

In the case of a driver driving mode, the memory 120 further stores asecond program for determining a head-up display (HUD) control commandusing the driver's state data including driver's gaze positioninformation and the driving situation data including position and colorinformation of a surrounding object, and the processor 130 executes thesecond program and transmits an HUD control command in consideration ofthe color information of the surrounding object displayed in an HUD areain the driver's gaze.

The processor 130 contrasts/compares a color of content displayed in theHUD area with a color of a surrounding object to transmit the HUDcontrol command to maximize day/night visibility/readability.

For example, the color of the content displayed in the HUD area and thecolor of the surrounding object are set in a complementary colorrelationship.

The processor 130 determines the gaze point using the driver's gazeposition information and transmits a control command to display HUDinformation at the gaze point.

In the case of a driver driving mode, the memory 120 further stores athird program for determining an HUD control command by using at leastone of the driver's state data related to a driver fatigue and thedriving situation data related to illuminance, and the processor 130executes the third program and transmits the HUD control command forchanging at least one of color, brightness, and saturation of contentdisplayed on the HUD.

Hereinafter, in order to help those skilled in the art to understand, anembodiment (first embodiment) for determining a driving pattern in anautonomous driving mode, an embodiment (second embodiment) forcontrolling an HUD in consideration of a driver's gaze, and anembodiment (third embodiment) for controlling the HUD in considerationof a driver's condition and illuminance information will be described.

First Embodiment

FIG. 2 illustrates a vehicle control apparatus according to anembodiment of the present disclosure.

A vehicle control apparatus according to the embodiment of the presentdisclosure simultaneously acquires data such as expression, facialmovement, tension, or relaxation state of a driver to determine adriver's state or emotion through a camera or biometric sensor togetherwith vehicle behavior data (speed, acceleration, yaw, pitch, roll angle,steering angle, etc.) and longitudinal (acceleration, brake pressurecommand, etc.) and lateral (steering wheel angle/torque command, etc.)control data used for autonomous driving control.

According to the embodiment of the present disclosure, optimization isperformed by learning the driver's state or emotion according to anautonomous driving control pattern, predicting driving control patternsdesired by individual drivers based on the learned data, confirming thedriver's emotion according to the predicted driving control pattern, andperforming re-learning.

The vehicle control apparatus according to the embodiment of the presentdisclosure includes a data collection unit 210 that collects vehicledata and driver data, a data recording and determining unit 220 thatstores the collected data and determines the driver's state, a drivingpattern determining unit 230 that learns data and predicts a driver'spreferred driving pattern, and an autonomous driving control unit 240that performs autonomous driving according to the driving pattern.

The data collection unit 210 includes an autonomous driving datacollection unit 211 and a driver image data collection unit 212 and adriver biosignal data collection unit 213.

The autonomous driving data collection unit 211 collects, as vehicledata, information such as speed, acceleration, a yaw/pitch/roll angle,and a steering angle acquired from a steering angle sensor, a vehiclespeed sensor, an acceleration sensor, a gyro sensor, etc. mounted in avehicle.

The driver image data collection unit 212 collects data such as adriver's gaze, eye blinking, pupil diameter, face angle, face position,and expression acquired by a camera attached to a dashboard at a top ofa cluster.

A driver biometric data collection unit 213 collects data such as anelectrocardiogram, grip strength, respiration, and pulse acquired by abiosensor attached to a steering wheel.

The data collection unit 210 collects control values from the autonomousdriving control unit 240.

The data recording and determining unit 220 matches and storesdriving-related data collected by the autonomous driving data collectionunit 211, driver image and biometric data collected by the driver imagedata collection unit 212 or the driver biometric data collection unit213, the control value received from the autonomous driving control unit240 according to time sequence.

The data recording and determining unit 220 determines the driver'sstate or emotion according to predefined driver's state or emotiondetermination criteria.

The driving pattern determining unit 230 performs a learning processusing the data stored in the data recording and determining unit 220 andtransmits an optimized vehicle control value to the autonomous drivingcontrol unit 240.

The driver image data collection unit 212 collects a driver's reactionto the autonomous driving according to the vehicle control value againand updates and stores the autonomous driving data and the control valuetogether.

The driver biometric data collection unit 213 collects a change indriver's state (tension/relaxation, etc.) about the autonomous drivingaccording to the vehicle control value again and updates and stores theautonomous driving data and the control value together.

The driving pattern determining unit 230 determines the autonomousdriving control value to approximate the driver's preferred drivingpattern by learning the driver image or biometric data, vehicle data,and autonomous driving control value and transmits the determinedautonomous driving control value to the autonomous driving control unit240.

The autonomous driving control unit 240 controls the vehicle using theautonomous driving control value received from the driving patterndetermining unit 230.

In this case, when the autonomous driving control value is a value thatis greater or less than a safety reference value range, the autonomousdriving control unit 240 does not apply the value to the control butapplies a safety reference value to perform the autonomous drivingcontrol.

FIG. 3 illustrates autonomous driving improvement in consideration of adriver's state or emotion according to an embodiment of the presentdisclosure.

The driver image data (fear and disgust) or biometric data (tension andrelaxation) of a driver who feels anxious or unsatisfied with theautonomous driving pattern driven by the initial control value ofautonomous driving is collected and learned together with the vehicledata and the control value, and the autonomous driving control value iscalculated.

During the autonomous driving according to the calculated autonomousdriving control value, the driver's reaction is fed back from the imageor biometric data, and the collection, learning, and control proceduresare consecutively repeated until the driver's preferred autonomousdriving pattern is approached.

As described above, the processor 130 may perform learning by matchingthe autonomous driving data, the driving situation data, and thedriver's state or emotion data according to time sequence and use thelearned results to transmit an autonomous driving control commandaccording to the driver's preferred driving pattern.

In addition, the processor 130 includes at least one of a steering wheelcontrol command for a turning radius on a curved road, a vehicle speedcontrol command, an acceleration control command, and a command formaintaining a separation distance from the front vehicle.

FIG. 4 illustrates a vehicle control method according to an embodimentof the present disclosure.

A vehicle control method according to the embodiment of the presentdisclosure includes collecting the driver image and biometric data, thedriving-related data, and the autonomous driving control value (S410),determining a driving pattern by confirming a driver preference pattern(S420), and performing the autonomous driving control according to thedetermined driving pattern (S430).

In operation S410, image data including the driver's expression andfacial movement to determine the driver's emotion and biometric dataincluding changes in electrocardiogram, grip strength, respiration, andpulse are collected.

In operation S410, driving-related data including speed, acceleration,yaw, pitch, a roll angle, and a steering angle is collected.

In operation S410, the autonomous driving control values including thelongitudinal control data and the lateral control data used forautonomous driving control are collected.

In operation S420, the driving-related data, the driver image orbiometric data, and the autonomous driving control values are matchedand stored according to the time sequence, and the autonomous drivingpattern is determined according to the driver's condition or emotiondetermination criteria.

In operation S430, the autonomous driving is performed according to thedetermined autonomous driving pattern, the driver's reaction iscollected again, and the autonomous driving control value and thedriving-related data are matched and updated.

Second Embodiment

A vehicle control system according to a second embodiment of the presentdisclosure controls a head-up display (HUD) in consideration of driver'sface (head) direction and gaze information, includes an input unit 110that receives the driver's face orientation and gaze information, amemory 120 that stores a program for determining an HUD informationdisplay position using the driver's face orientation and gazeinformation, and a processor 130 that executes the program, in which theprocessor 130 determines a gaze point using the driver's faceorientation and gaze information and determines a display position todisplay the HUD information at the gaze point when there is nopossibility of overlap with safety-related essential information of theHUD.

The processor 130 monitors the driver's face orientation and gazeinformation received through the input unit 110 according to timesequence by linking whether there is a front vehicle, the possibility ofoverlapping a position, or color of a lighting device with an HUD imagecolor with driving information to determine whether the driver's gazechange is a gaze change in situation in which the display position needsto change.

When it is determined that the display position does not need to change,the processor 130 determines that the driver's gaze point is maintainedas a front area and maintains the display position.

When it is determined that the display position needs to change, theprocessor 130 determines that the driver's current field of view is thegaze point and changes the display position so that the HUD informationis displayed at the gaze point.

When it is determined that the display position needs to change, theprocessor 130 determines that the driver's current field of view is thegaze point and changes the display position so that the HUD informationis displayed at a position close to the gaze point.

In addition, when the information provided at the changed displayposition and the color of the surrounding environment at the positionthat the driver is looking at are similar, since the informationtransfer effect may be inhibited, the processor secures visibility andreadability (readability) by changing a background color of displayedinformation.

FIG. 5A and FIG. 5B illustrate an example of a change in HUD informationdisplay area in consideration of driver's field of view informationaccording to another embodiment of the present disclosure.

Referring to FIG. 5A and FIG. 5B, when the driver's field of view is a Vdirection and the driver does not look ahead, the HUD information isdisplayed on right windshield area A in consideration of the driver'sfield of view.

In this case, by accumulating and monitoring the driver's faceorientation and gaze information according to time sequence, theprocessor 130 determines whether the change in the driver's gazecorresponds to a gaze change in which the HUD information displayposition is to be changed.

For example, it is assumed that a vehicle traveling on a three-lane roadof one-way needs to make a right turn at 100 meters ahead (navigationinformation is used).

In this case, in the situation where a vehicle enters the rightmost laneand is traveling, when a driver looks at a right area instead of thefront at a point where he/she should turn right, this is a gaze changethat occurs because the driver has already received navigationinformation from the HUD information and is ready to turn right (thatis, a situation in which the driver's gaze point is changed to preventaccidents that may occur to pedestrians or the like when turning right).

Then, it can be said that a driver is highly likely to look ahead(front) again after making a right turn. When the HUD informationdisplay position is changed and displayed in the driver's gaze pointarea (right side) when making a right turn and the driver's gaze pointarea (front) is determined after making a right turn again, and the HUDinformation is displayed in the front area accordingly, the HUDinformation may be displayed by moving to the area at a later time,which may cause discomfort of a user.

Therefore, according to another embodiment of the present disclosure,instead of simply changing the HUD information display position inconsideration of the driver's field of view information, the drivingsituation information and the driver's gaze change information aremonitored, and thus, the HUD information display position can change atthe necessary moment.

Conversely, in the above-described example, it is assumed that a vehicletraveling on a three-lane road of one-way needs to make a right turn 100meters ahead (navigation information is used).

In this case, even though there is not much distance left to make aright turn, when a driver is driving in a second lane and shows nointention to change lanes, such as turning on a right turn signal, theHUD information changes a color, a size, a display method, or the like(for example, displayed in a flickering manner) to inform a driver thata right turn is required according to the navigation information.

In this case, when the driver looks at the right area, it is determinedthat the driver intends to make a right turn, and as described above,the driver does not change the HUD information display positionseparately because it is expected that the driver will look ahead againafter making a right turn.

As described above, even if the HUD information is provided by changingthe display method for more than a certain period of time, when thedriver's field of view information does not change (that is, when thedriver continues to look ahead), it is determined that the driverintends to drive on a route different from the route guided by thenavigation system, and the HUD information is displayed in a normal modeas in the previous method.

When the driver is looking at an object of interest (a nearby building,a new vehicle driving in the next lane, etc.) in a signal waitingsituation, the processor changes the HUD information display position tothe driver's gaze point.

Through this, when vehicle departure notification and traffic lightchange notification information are provided through the HUDinformation, it is possible to increase driving convenience byrecognizing the HUD information in which the display position isdetermined in response to the driver's gaze point even if the driverdoes not look at the fixed HUD display area.

When the driver's gaze point changes to directions other than forward,the processor 130 provides a convenience function to the driver by usingthe driver's pupil movement or other body's gestures.

For example, in a signal waiting situation, when the driver is lookingat a building where a bakery shop is newly opened and people are liningup, the processor maintains and displays driving-related information inthe existing HUD display area, and displays the information of theobject by matching the information of the object viewed by the driver tothe area of the gaze point (for example, business name, phone number,menu, etc.).

In this case, when the driver blinks his or her eye twice or inputs ahand gesture in the form of a check mark, the processor transmits andstores the information of the object of interest to the device possessedby the user or the device inside the vehicle (for example: OOO bakeryregistered as a location of interest on OO, OO, 2020, location:OOOO-crossroad, OO, OO-ro, OO-si, Seoul).

In the future, when the location of the object of interest is located onthe driver's route, it is possible for the processor to provide anotification to the driver (for example, provide voice notificationsaying “OOO bakery previously registered as a location of interest is onthe way. The main menu OOO will come out at 10 am, 1 pm, and 4 pm, andwe will pass by this location around 1:10 pm. When you want to visit OOOBakery, enter the OK gesture and we will register it as a stopover.”).

FIG. 6 illustrates a vehicle control method according to anotherembodiment of the present disclosure.

A vehicle control method according to another embodiment of the presentdisclosure includes tracking driver's face orientation and gazeinformation (S610), determining the HUD information display positionusing the driver's face orientation and gaze information (S620) anddisplaying the HUD information according to the determined displayposition (S630).

In operation S620, it is determined whether it is necessary to changethe HUD information display position according to the change in the gazeinformation by linking and monitoring whether there is a front vehicle,the possibility of overlapping a position or color of a lighting devicewith an HUD image color, and the change in the face orientation and gazeinformation with the driving situation.

In operation S620, when it is determined that the driver's gaze point ischanged according to the change in the gaze information, the HUD displayposition is changed so that the area corresponding to the gaze pointbecomes the HUD information display position.

In operation S620, when the driver's gaze temporarily deviates from thefront according to the change in the gaze information but his or hergaze is expected to return to the front according to the drivingsituation, the HUD information display position is maintained withoutchanging.

In operation S610, the driver's face orientation and gaze information isreceived from a vehicle internal sensor, such as a driver's eye trackerand a camera sensor.

In this case, in operation S610, the driving information (for example,navigation information) is received along with the driver's faceorientation and gaze information, and a process of changing the driver'sface orientation and gaze information is monitored.

In operation S620, according to the result of monitoring the process ofchanging the driver's face orientation and gaze information, it isdetermined whether the situation according to the change of the driver'sgaze point is a situation in which the HUD information display positionneeds to be changed.

When it is determined in operation S620 that the driver's faceorientation and gaze point change situation is not a situation in whichthe HUD information display position needs to change, in operation S630,the HUD information is continuously displayed to the previouslydisplayed position.

For example, it is assumed that a vehicle traveling on a three-lane roadof one-way needs to make a right turn at 100 meters ahead (navigationinformation is used).

In this case, in the situation where a vehicle enters the rightmost laneand is traveling, when a driver looks a right area instead of the frontat a point where he/she should turn right, this is a gaze change thatoccurs because the driver has already received navigation informationfrom the HUD information and is ready to turn right (that is, asituation in which the driver's gaze point is changed to preventaccidents that may occur to pedestrians or the like when turning right).

Then, it can be said that a driver is highly likely to look ahead(front) again after making a right turn. When the HUD informationdisplay position is changed and displayed in the driver's gaze pointarea (right side) when making a right turn and the driver's gaze pointarea (front) is determined after making a right turn again, and the HUDinformation is displayed in the front area accordingly, the HUDinformation may be displayed by moving to the area at a later time,which may cause discomfort of a user.

On the other hand, in operation S620, when it is determined that thesituation in which the driver's face orientation and gaze informationchanges is to change the HUD information display position, for example,in the signal waiting situation, in the case where the driver does notlook at movement of a front vehicle or a traffic light but looks atother locations, in operation S620, it is determined to display the HUDinformation (for example, front vehicle departure notification andchange notification of a traffic light) in the driver's gaze point area,and in operation S630, the HUD information is displayed at the changeddisplay position.

Third Embodiment

A vehicle control system according to another embodiment of the presentdisclosure includes an input unit 110 that receives driver's stateinformation and illuminance information, a memory 120 that stores aprogram for determining an HUD display method by using at least one ofthe driver's state information and the illuminance information, and aprocessor 130 that executes the program, in which the processor 130controls to change and display a color or saturation of the HUDinformation using at least one of the driver's state information and theilluminance information.

The processor 130 controls to change and display hue, saturation, andluminance of a head-up display (HUD) information using RGB colorinformation as well as simple vertical and horizontal illuminanceinformation.

The processor 130 controls to change and display the hue, saturation,and luminance of the HUD information in consideration of the RGB colorinformation (points) or distributions (planes) at a position at which adriver is looking.

The input unit 110 receives state information related to a driverfatigue from a driver monitoring camera, a handle grip pressure sensor,and a heart rate sensor.

The input unit 110 receives illuminance and weather information around avehicle through a camera sensor outside the vehicle.

In this case, the processor 130 grasps color information usingaccumulated surrounding information as well as instantaneous surroundinginformation (headlight light of an oncoming vehicle, surroundingillumination, street light, etc.) received through the input unit 110,and controls to change and display the color, saturation, or luminanceof the HUD information.

In addition, the processor 130 can use color, saturation, and luminancechange display history information of HUD information received fromanother vehicle driving ahead along a current driving route topreemptively perform HUD control based on the expected surroundinginformation based on position information as well as HUD controlcorresponding to the surrounding information received through the inputunit 110.

The input unit 110 processes the illuminance information that affects adriver's field of view in consideration of vehicle's tinting through thein-vehicle camera sensor and transmits the processed illuminanceinformation to the processor.

Since the external strong light gives a strong intensity of stimulus tothe driver with high fatigue, the processor 130 controls to change theHUD information to a color with a weak stimulus and to brightness andluminance considering visibility and readability and display the HUDinformation.

When it is determined that the intensity of external light is strongerthan a preset value by analyzing the illuminance information, theprocessor 130 controls to change the HUD information to change to acolor with relatively low saturation and a color with high luminance anddisplay the HUD information.

When it is determined that the driver fatigue is greater than or equalto a preset value and the intensity of external light is greater than orequal to the preset value, the processor 130 controls to lower thesaturation of the preset color to a certain level in order to reduce thedriver fatigue, and change the HUD information to an appropriate levelof luminance and display the HUD information.

The above-described illuminance information includes not onlyilluminance according to weather but also horizontal plane illuminanceand vertical illuminance such as headlight light by an oncoming vehicleand illuminance by a street light.

According to another embodiment of the present disclosure, it ispossible to identify a driver's state through a camera and a biosignaldevice inside the vehicle and display information by correcting thecolor through the HUD by determining the color, saturation, andluminance suitable for the driver fatigue.

According to another embodiment of the present disclosure, thesaturation and luminance of the HUD color are controlled inconsideration of the intensity of light flowing into the vehicleaccording to the illuminance and weather information of the vehicledriving environment.

According to another embodiment of the present disclosure, it ispossible to reduce the driver fatigue according to the HUD changedisplay and more accurately support the HUD information.

FIG. 7A-FIG. 7C illustrate a concept of the HUD control using thedriver's state and the surrounding information according to stillanother embodiment of the present disclosure.

The processor 130 uses the driver's state information as illustrated inFIG. 7A and uses the illuminance information as illustrated in FIG. 7Bto determine the HUD display method as illustrated in FIG. 7C.

As described above, the processor 130 controls to change the HUDinformation to a color with a weak stimulus to be displayed to a driverwith high fatigue, and controls to change the HUD information to a colorwith a relatively low saturation and an appropriate level of luminanceand display the HUD information when it is determined that the intensityof external light (illuminance information including weather, streetlights, headlights of an oncoming vehicle, etc.) is stronger than thepreset value using the illuminance information.

FIG. 8 illustrates a vehicle control method according to yet anotherembodiment of the present disclosure.

A vehicle control method according to yet another embodiment of thepresent disclosure includes determining the driver's state (S810),determining the external environment information (S820), and controllingat least any one of the color, brightness, and position of the HUD usingat least one of the driver's state and the external environmentinformation (S830).

In operation S810, state information related to a driver fatigue isreceived from a driver monitoring camera, a handle grip pressure sensor,and a heart rate sensor.

In operation S820, the illuminance and weather information around thevehicle are received through the camera sensor outside the vehicle, andthe illuminance information that affects the driver's field of view isprocessed in consideration of the vehicle's tinting through the camerasensor inside the vehicle to determine the external environmentinformation.

In operation S830, since strong external light gives a strong intensityof stimulus to a driver with high fatigue, the HUD information changesto a color with a weak stimulus and is displayed.

In operation S830, when it is determined that the intensity of externallight is stronger than a preset value by analyzing the illuminanceinformation, the HUD information changes to a color with relatively lowsaturation and is displayed.

In operation S830, when it is determined that the driver fatigue isgreater than or equal to a preset value and the intensity of externallight is greater than or equal to the preset value, the saturation ofthe preset color is lowered to a certain level in order to reduce thedriver fatigue, and the HUD information changes to an appropriate levelof luminance and is displayed.

According to another embodiment of the present disclosure, when it isdetermined that the intensity of light (refer to horizontal planeilluminance) of a certain portion is stronger than a preset value byanalyzing the illuminance information (for example, the situation causedby the headlight of an oncoming vehicle), the driver's face is directedforward, and the HUD information is controlled to be displayed at aposition where it is determined that there is relatively little visualfatigue which allows a driver to look at a driving lane as much aspossible.

Meanwhile, the vehicle control method according to the embodiment of thepresent disclosure may be implemented in a computer system or recordedin a recording medium. The computer system may include at least oneprocessor, a memory, a user input device, a data communication bus, auser output device, and storage. Each of the above-described componentsperforms data communication through the data communication bus.

The computer system may further include a network interface coupled to anetwork. The processor may be a central processing unit (CPU) or asemiconductor device that processes instructions stored in the memoryand/or storage.

The memory and storage may include various types of volatile ornon-volatile storage media. For example, the memory may include a readonly memory (ROM) and a random access memory (RAM).

Accordingly, the vehicle control method according to the embodiment ofthe present disclosure may be implemented as a computer-executablemethod. When the vehicle control method according to the embodiment ofthe present disclosure is performed in a computer device, computerreadable instructions may perform the vehicle control method accordingto the present disclosure.

Meanwhile, the vehicle control method according to the presentdisclosure described above may be implemented as a computer readablecode on a computer-readable recording medium. The computer-readablerecording medium includes any type of recording medium in which datareadable by a computer system is stored. For example, there may be aROM, a RAM, a magnetic tape, a magnetic disk, a flash memory, an opticaldata storage device, and the like. In addition, the computer-readablerecording medium may be distributed in computer systems connectedthrough a computer communication network, and stored and executed asreadable codes in a distributed manner.

According to the present disclosure, it is possible to resolvediscomfort and anxiety of drivers about autonomous driving and toimprove drivers' feelings of reliability for the autonomous driving bymonitoring drivers' reactions to autonomous driving patterns in realtime, calculating autonomous driving patterns optimized for each driver,and performing the autonomous driving accordingly.

It is possible to focus on services provided in various aspects andimprove driver's convenience by improving drivers' feelings ofreliability for autonomous driving.

In addition, when authentication and cloudization of individual driversare performed, it is possible to improve driver satisfaction byproviding autonomous driving patterns optimized for each driver nomatter what autonomous vehicle is used.

According to the present disclosure, it is possible to support intuitiverecognition of HUD information even when a driver looks at locationsother than the front by moving the HUD information to a driver's gazepoint.

According to the present disclosure, it is possible to control HUDinformation to be changed and displayed so as to reduce the driverfatigue and improve visibility/readability in response to the externalenvironment.

The effects of the present disclosure are not limited to theabove-described effects, and other effects that are not mentioned may beobviously understood by those skilled in the art from the followingdescription.

What is claimed is:
 1. A vehicle control system comprising: an inputunit configured to collect driving situation data, infrastructure data,forward situation data, and driver's state data; a memory configured tostore a program for determining a driving pattern using the drivingsituation data, the infrastructure data, the forward situation data, andthe driver's state data in a case of an autonomous driving mode; and aprocessor configured to execute the program, wherein the processorperforms learning by matching the driving situation data, theinfrastructure data, the forward situation data, and the driver's statedata according to time sequence, uses learned results to determine adriver's preferred autonomous driving control pattern, and transmits anautonomous driving control command according to the autonomous drivingcontrol pattern.
 2. The vehicle control system of claim 1, wherein thedriving situation data includes speed, acceleration, yaw, pitch, androll information and steering angle and ADAS(Advanced Driver AssistanceSystems) information of a vehicle.
 3. The vehicle control system ofclaim 1, wherein the driver's state data includes gaze directioninformation, eye blink information, pupil diameter information, gazeinformation, face position information, and expression information of adriver.
 4. The vehicle control system of claim 1, wherein the input unitacquires the infrastructure data including information on weather, apavement condition, and a road structure.
 5. The vehicle control systemof claim 1, wherein the input unit acquires the forward situation datathat includes information on whether a front vehicle is present, a colorand a rear lamp and a size of the front vehicle, and whether a gazepoint overlaps a head-up display (HUD) position and a lighting device ofthe front vehicle.
 6. The vehicle control system of claim 1, wherein theprocessor includes at least one of a steering wheel control command fora turning radius on a curved road, a vehicle speed control command, anacceleration control command, a command for maintaining a separationdistance from a front vehicle, and a command for maintaining a lateralseparation distance.
 7. The vehicle control system of claim 1, wherein,in a case of a driver driving mode, the memory stores a second programfor determining a head-up display (HUD) control command using thedriver's state data including driver's gaze position information and thedriving situation data including position and color information of asurrounding object, and the processor executes the second program andtransmits the HUD control command in consideration of the colorinformation of the surrounding object displayed in an HUD area on adriver's gaze.
 8. The vehicle control system of claim 7, wherein theprocessor compares a color of content displayed in the HUD area with thecolor of the surrounding object and transmits the HUD control commandfor changing at least one of color, luminance, and brightness of ahead-up display user interface (HUD UI) and a size of the content inconsideration of visibility for each driving time.
 9. The vehiclecontrol system of claim 7, wherein the processor determines a gaze pointusing the driver's gaze position information and transmits a controlcommand to display HUD information at the gaze point.
 10. The vehiclecontrol system of claim 1, wherein, in a case of a driver driving mode,the memory stores a third program for determining a head-up display(HUD) control command by using at least one of the driver's state datarelated to a driver fatigue and the driving situation data related toilluminance, and the processor executes the third program and transmitsthe HUD control command for changing at least one of color, brightness,and saturation of content displayed on the HUD.
 11. A vehicle controlmethod comprising: (a) collecting driver's state data and driving statedata; (b) in an autonomous driving mode, performing learning usingautonomous driving control data, the driver's state data, and thedriving state data, and determining a driver's preferred drivingpattern; and (c) transmitting an autonomous driving control commandaccording to the driving pattern.
 12. The vehicle control method ofclaim 11, wherein, in operation (a), the driving situation dataincluding speed, acceleration, yaw, pitch, and roll information andsteering angle and ADAS (Advanced Driver Assistance Systems) informationof a vehicle, and the driver's state data including gaze directioninformation, eye blink information, pupil diameter information, gazeinformation, face position information, and expression information of adriver are collected.
 13. The vehicle control method of claim 11,wherein, in operation (b), the driving pattern including at least one ofa steering wheel control command for a turning radius on a curved road,a vehicle speed control command, an acceleration control command, and acommand for maintaining a separation distance from a front vehicle isdetermined.
 14. The vehicle control method of claim 11, furthercomprising, (d) when the autonomous driving mode is switched to a driverdriving mode, acquiring driver's gaze position information andinformation on a position, a color, and luminance of a surroundingobject corresponding to the position, and determining a head-up display(HUD) control command.
 15. The vehicle control method of claim 14,wherein, in operation (d), a color of content displayed in an HUD areais compared with the color of the surrounding object in consideration ofthe color information of the surrounding object displayed in the HUDarea in the driver's gaze and transmits the HUD control command forchanging a color of a head-up display user interface (HUD UI) inconsideration of visibility for each driving time.
 16. The vehiclecontrol method of claim 11, further comprising, (e) when the autonomousdriving mode is switched to a driver driving mode, determining a head-updisplay (HUD) control command by using at least one of the driver'sstate data related to a driver fatigue and the driving situation datarelated to illuminance.
 17. The vehicle control method of claim 16,wherein, in operation (e), the HUD control command for changing at leastone of color, brightness, and saturation of content displayed on the HUDis determined.